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Introduction 


Cervical  cancer  is  theoretically  completely  preventable  by  effective  screening  using  cervical  cytology  methods 
(the  Pap  test).  The  process  of  preparing  and  interpreting  Pap  tests  remains  one  of  the  last  high-volume  manual 
processes  in  the  clinical  laboratory.  Recent  technological  advances  in  specimen  preparation  and  computerized 
primary  screening  make  automated  approaches  to  cervical  cancer  screening  possible.  In  addition,  advances  in 
information  technology  have  facilitated  the  Internet  transmission  and  archival  storage  of  digital  images  and  other 
clinical  information.  The  combination  of  automated  preparation  and  screening  of  cervical  cytology  specimens,  with 
Internet  transmission  of  selected  images,  and  remote  interpretation  and  reporting  of  results  has  not  been  previously 
attempted. 

This  project  develops  a  highly  automated  cervical  cytology  screening  system,  a  software  interface  capable  of 
transmitting  and  presenting  images  to  remote  reading  stations,  with  facility  for  immediate  results  reporting  back  to 
the  specimen  source.  Clinical  studies  utilizing  this  developed  system  will  be  performed  to  test  accuracy  and 
functionality  against  the  current  on-site  manual  screening  process.  Primary  development  of  the  system  has  been 
accomplished  at  the  Massachusetts  General  Hospital  (MGH)  site  and  reading  stations  have  been  installed  at  MGH 
and  at  Walter  Reed  Army  Medical  Center  (WRAMC).  A  phase  1  pilot  study  has  been  completed,  data  has  been 
analyzed  and  reported  (see  2008  publication  in  American  Journal  of  Clinical  Pathology  -  attached).  Patient  accrual 
for  the  phase  2  study  targeting  500  prospectively  obtained,  consented  patients  has  been  completed  (356  patients 
finally  enrolled)  following  a  very  lengthy  delay  due  to  US  Army  1RB  oversight  approval  processes  (Office  of 
Research  Protections)  as  reported  in  the  2007  Annual  Report.  Reading/reporting/ Analysis  of  this  phase  will  be 
complete  in  July  2008.  Preplanning  for  the  final  phase  3  clinical  trial  to  be  performed  with  the  121st  Army  Hospital 
in  Seoul,  Korea  is  near  complete  with  an  IRB  submission  planned  for  summer  of  2008  and  patient  accrual  to  begin 
as  soon  as  approval  is  granted. 
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Body 


The  following  is  a  summation  of  the  work  completed  to  the  present  time  based  on  the  project’s  accepted  Statement 
of  Work.  Details  follow  below  in  an  expanded  version  of  the  Statement  of  Work: 

Statement  of  Work 


Task  1:  Complete  hardware,  software  and  network  development  required  for  testing  of  the  internet-based 
cervical  cytology  screening  system 

a)  Modify  the  FocalPoint  device  to  accept,  process  and  analyze  ThinPrep  specimens  -  completed 

b)  Adapt  FocalPoint  hardware  for  internet  transmission  of  digital  images  from  ThinPrep  and  SurePath 
specimens  -  completed 

c)  Adapt  commercial  software  (Wellogic)  to  permit  rapid  and  secure  transmission  of  digital  images  to 
remote  review  stations  -  completed 

d)  Procure  and  install  remote  microscopy  stations  (2)  -  completed 

e)  Adapt  commercial  software/hardware  (Wellogic)  to  allow  secure,  automated  reporting  of  cervical  cancer 
screening  results  -  completed 

f)  Adapt  commercial  software  (Wellogic)  to  integrate  screening  results  reporting  with  medical  decision 
support  system  -  All  Phase  2  modifications  have  been  made  ans  analyzed  for  proper  functionality. 

g)  perform  initial  testing  of  integrated  hardware/software/network  -  completed 

Task  2:  Develop  morphology  and  terminology  for  digital  images  and  perform  pilot  clinical  trial 

a)  Develop  a  set  of  learning  cases  with  known  diagnostic  outcome  -  pilot  set  of  200  cases  completed  (100 
SurePath,  100  ThinPrep). 

b)  Develop  morphologic  criteria  for  accuracy  of  interpretation  -  Phase  1  completed,  pending  Phase  2 
modifications  (July  2008). 


6 


c)  Develop  reporting  terminology  appropriate  for  case  management  -  Phase  1  completed,  pending  Phase  2 
modifications  (July  2008). 

d)  Develop  medical  decision  support  algorithms  -  Phase  1  completed,  pending  Phase  2  modifications 
(July  2008). 

e)  Perform  pilot  trial  using  a  set  of  500  unknown  specimens  to  identify  preliminary  system  performance 
characteristics  -  This  is  a  Phase  2  task.  Patient  accrual  has  been  completed  at  MGH  and  Walter  Reed 
sites. 

f)  Modify  procedures/equipment  based  on  pilot  trial  results  -  Phase  2  modifications  completed. 

g)  Develop  training  methods/materials  for  clinical  practice  -  Phase  2  modifications  completed. 

Task  3:  Complete  large,  prospective  clinical  trial  of  the  performance  of  the  internet-based  system  compared 
to  conventional  on-site  screening. 

a)  Develop  and  receive  approval  for  clinical  trial  protocol  and  consent  forms  -Phase  3  protocols  required 
modification  based  on  information  learned  to  date  in  phase  2.  Final  protocol  is  currently  being 
prepared  for  submission  to  local  IRBs  and  ORP.  (planned  submission  summer  2008) 

b)  Install  equipment  at  selected  sites  -  Completed  at  MGH  and  Walter  Reed  sites.  Installation  at  121st 
Army  Hospital  will  await  IRB  approval.  Preplanning  for  this  installation  has  been  investigated  and 
all  necessary  arrangements  have  been  made  to  implement  immediately  upon  receipt  of  necessary 
regulatory  approvals. 

c)  Train  clinical  personnel  participating  at  selected  sites  -  phase  3  task 

d)  Conduct  the  clinical  trial  -  phase  3  task 

e)  Perform  trial  data  analysis  -  phase  3  task 

f)  Prepare  report  of  trial  with  implementation  recommendation  -phase  3  task 

Expanded  Discussion 
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A)  Phase  1  of  the  project  has  been  completed.  This  Phase  included: 

1)  development  of  hardware,  software,  and  interfaces  between  computerized  scanning  device  and  Internet- 
linked  servers  and  reading  stations. 

2)  development  of  a  200  case  test  set  of  slides  with  known  reference  diagnosis  (100  SurePath  and  100 
ThinPrep  slides) 

3)  analysis  of  the  test  set  on  the  prototype  system  with  interpretation  by  6  individuals  (3  cytotechnologists, 

3  pathologists) 

4)  data  analysis 

5)  reporting  of  the  data  in  3  abstracts  presented  at  the  US-Canadian  Academy  of  Pathology  Annual  meeting 
(February  2006) 

6)  development  of  training  materials  to  guide  and  improve  performance 

7)  submission  of  revisions/improvements  to  software 


Comments:  Phase  1  showed  a  successful  first  feasibility  trial  of  this  system.  191  cases  were  included  in  the  analysis 
(SP-101,  TP-90;  99-N1LM,  4-ASC-US,  3-ASC-H,  4-AGC,  63-LSIL,  18-HSIL).  >3  reviewers  agreed  on  the  correct 
general  categorization  for  unsatisfactory/normal  in  87%,  and  for  abnormal  in  83%.  For  specific  Bethesda 
interpretation,  >3  reviewers  agreed  on  the  correct  categorizations  as  follows:  ASC-US  -  75%,  ASC-Fl  -  100%,  AGC 
-  25%,  LSIL  -  83%,  HSIL  -  94%.  These  results  indicate  that  correct  triage  of  abnormal  cases  could  be  performed  at 
a  sensitivity  very  comparable  to  the  manual  screening  standard.  In  addition  it  was  noted  during  the  data 
analysis/training  phase,  that  a  substantial  number  of  the  "missed"  cases  had  to  do  with  experience  of  the  observers  in 
identifying  clues  present  in  the  review  station  images  or  with  institutional  "biases,"  meaning  differences  in 
interpretations  that  could  be  traced  to  practice  setting  differences  between  MGH  and  WRAMC.  A  new  publication 
detailing  the  results  of  the  successful  phase  1  trial  has  been  published  as  the  "lead"  article  in  the  American  Journal  of 
Clinical  Pathology,  (pdf  of  article  attached)  The  reference  is: 

Eichhorn  JFI,  Buckner  L,  Buckner  SB,  Beech  DP,  Harris  KA,  McClure  DJ,  Crothers  BA,  Wilbur  DC. 

Internet-based  gynecologic  telecytology  with  remote  automated  image  selection:  results  of  a  first-phase 
developmental  trial.  Am  J  Clin  Pathol.  2008  May;129(5):686-96. 


B)  Phase  2  of  the  project  is  nearly  complete 

After  significant  delays  as  detailed  in  the  2007  Annual  report  and  reiterated  below,  the  phase  2  patient  accrual  has 
been  completed.  With  a  target  of  500  patients  between  MGH  and  Walter  Reed  sites,  356  were  finally  enrolled.  It 
was  felt  by  the  site  investigators  that  this  accrual  was  sufficient  to  perform  the  trial  adequately  and  it  was  necessary 
to  close  the  enrollment  due  to  the  difficulty  in  consenting  patients  at  the  respective  busy  practice  clinics.  Wellogic 
reading/reporting  system  modifications  required  significant  work  as  the  company  had  migrated  their  information 
systems  to  a  new  operating  system  version  platform  since  phase  1  work  had  been  completed.  The  reading/reporting 
package  required  "migration"  to  this  new  platform  via  a  series  of  software  modifications  and  trials.  This  process  was 
complicated  by  the  "glitches"  that  developed  (as  might  be  expected  in  a  software  upgrade)  which  required  the 
database  to  be  purged  and  reloaded  multiple  times.  Patients  enrolled  have  now  been  entered  into  the  registration 


system,  slides  have  been  scanned  and  reconciliation  of  the  images  and  patient  demographic  information  has  been 
completed.  Reading  of  the  specimens  followed  by  data  analysis  will  take  place  by  July  2008. 

Phase  2  timeline 

1)  Local  1RB  approvals  were  granted  for  Phase  2  at  MGH  and  WRAMC 

2)  US  Army  "oversight"  IRB  (Office  of  Research  Protections  -  ORP)  required  review  submitted  - 
5/17/2005. 

3)  ORP  requested  revisions  -  1/3/2006 

4)  Revisions  submitted  to  ORP  -  2/20/2006 

5)  ORP  final  approval  for  WRAMC  was  granted  on  8/3/07. 

6)  ORP  final  approval  for  MGH  was  granted  on  2/12/07. 

7)  Patient  accrual  completed  on  1 1/30/07. 

8)  Software  modifications  to  the  Wellogic  reading/reporting  system  completed  on  4/15/08. 

9)  Patients  and  specimens  accessioned  to  the  system  completed  on  5/20/08. 

10)  Study  staff  retraining  to  take  place  (6/1-6/30/08) 

11)  Reading  to  commence  7/1/08. 

12)  Analysis  to  be  completed  by  7/30/08. 


C)  Phase  3  changes  since  the  last  Annual  Report 

1)  The  Phase  3  protocol  is  being  prepared  at  the  time  of  this  report.  Significant  modifications  are  being 
planned  to  this  protocol  based  on  experienced  gained  in  phase  2.  Submission  planned  in  summer  of  2008. 

2)  An  assurance  was  obtained  for  research  to  be  performed  at  the  121st  Army  Hospital  in  Seoul,  Korea  via 
the  Tripler  Army  Medical  Center. 

3)  All  initial  arrangement  have  been  made  with  vendors  (Becton  Dickinson/TriPath,  Wellogic,  Zeiss)  to 
insure  that  installations  in  Korea  can  be  accomplished  rapidly  following  regulatory  approvals  for  the  phase 
3  protocol. 
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Key  Research  Accomplishments 


1)  Assurance  obtained  for  Phase  3  oversight  at  121st  Army  Hospital  via  Tripler. 

2)  Patient  accrual  for  phase  2  completed  at  MGH  and  Walter  Reed  sites. 

3)  Modifications  have  been  completed  for  the  reading/reporting  station  software  (Wellogic)  based  on  outcomes  of 
Phase  1. 

4)  All  issues  based  on  operating  system  platform  changes  (Wellogic)  have  been  resolved. 

5)  Preplanning  for  phase  3  protocol  development  is  near  complete  (summer  2008) 

6)  A  no  cost  annual  extension  has  been  approved  for  this  project,  (at  the  time  of  this  writing  we  have  received  email 
notification  of  this  extension  but  have  not  received  the  formal  documentation) 
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Reportable  Outcomes 


1)  Final  publication  of  pilot  study  publication. 

Eichhorn  JH,  Gelfand  JA,  Brauns  TA,  Crothers  B,  Wilbur  DC.  A  Novel  Automated  Screening  and 
Interpretation  Process  for  Cervical  Cytology  Using  Internet  Transmission  of  Low  Resolution  Images:  A  Pilot 
Study.  Cancer  (Cancer  Cytopathol)  2005;105:199-206 

2)  Publication  of  Phase  1  results  complete. 

Eichhorn  ML  Buckner  L,  Buckner  SB,  Beech  DP,  Harris  KA.  McClure  DJ,  Crothers  BA,  Wilbur  DC.  Internet- 

based  gynecologic  telecytology  with  remote  automated  image  selection:  results  of  a  first-phase  developmental 
trial.  Am  J  Clin  Pathol.  2008  May;129):686-96. 
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Conclusions 


1)  Phase  2  trial  is  nearing  completion  -  patient  accrual  has  been  completed;  all  patients  have  been  accessioned  into 
the  system  and  all  slide  have  been  scanned;  retraining  of  study  personnel  is  ongoing;  all  software  modifications  have 
been  completed. 

2)  Assurance  has  been  obtained  for  121st  Army  Hospital  in  Seoul,  Korea  via  Tripler. 

3)  Preplanning  for  phase  3  IRB  protocol  submission  is  near  complete. 

4)  All  arrangements  are  in  place  for  immediate  placement  of  instrumentation  at  the  Korea  site  following  regulatory 
approvals  of  the  phase  3  protocol. 
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Appendices/Attachments 


1)  PDF  file  of  the  American  Journal  of  Clinical  Pathology  2008  lead  article  detailing  the  phase  1  results. 
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Abstract 

A  retrospective  set  of  191  gynecologic  cytology 
slides  with  reference  interpretations  was  run  on  an 
automated  screen  ing  device  that  selects  fields  of 
view  (FOVs)  based  on  a  hierarchical  probability 
of  abnormality  being  present.  An  interface  was 
developed  between  the  device  and  a  remote  server 
using  customized  image  review  software.  FOVs  were 
reviewed  by  3  cytotechnolo gists  and  3  cytopatholo gists, 
and  binary  triage  (unsatisfactory  for  evaluation/ 
negative  for  intraepithelial  lesion  or  malignancy 
[NILM]  vs  “ abnormal  ”  [neither  unsatisfactory  nor 
N1LM] )  and  specific  interpretations  were  done.  No 
morphologic  training  before  FOV  review  was  provided. 

Three  or  more  reviewers  agreed  on  the  correct 
categorization  of  NILM/unsatisfactory  in  89%  (85/96) 
and  abnormal  in  83%  (79/95).  Three  or  more  reviewers 
triaged  cases  to  abnormal  as  follows:  atypical 
squamous  cells  of  uncertain  significance,  83%  (5/6); 
atypical  squamous  cells,  cannot  exclude  high-grade 
lesion,  100%  (3/3);  low-grade  squamous  intraepithelial 
lesion  (S1L),  83%  (52/63);  high-grade  SIL,  94% 
(17/18);  and  atypical  glandular  cells,  40%  (2/5). 

This  procedure  may  have  comparable  sensitivity 
and  specificity  and  possibly  could  provide  effective 
initial  triage  to  further  evaluation,  A  review  of 
individual  cases  suggests  that  further  accuracy  can  be 
achieved  with  additional  training  and  experience. 


We  previously  hypothesized  that  the  melding  of  2  tech¬ 
nologies,  the  Internet  and  computerized  automated  scanning, 
could  lead  to  the  provision  of  effective  cervical  cancer  screen¬ 
ing  programs  for  countries  lacking  a  trained  cytology  work¬ 
force  and  to  an  efficient,  economical,  and  centralized  way 
of  triaging  specimen  data.  In  a  pilot  study,  we  showed  that 
satisfactory  and  reliable  interpretations  of  device-selected, 
electronically  transmitted  images  can  be  made  on  a  computer 
monitor,  without  manual  microscopy.1 

The  instrument  used  in  that  study,  the  FocalPoint  screen¬ 
ing  system  for  cervical  cytology  (Becton  Dickinson/TriPath 
Imaging,  Burlington,  NC),  is  used  in  many  laboratories  to 
triage  cases  to  no  further  review  or  complete,  manual  review. 
Studies  have  shown  that  the  use  of  this  automated  device 
allows  cytologists  to  efficiently  and  reliably  identify  more 
abnormal  cases  than  manual  screening  alone.2’4  This  device 
also  has  3  features  that  make  possible  the  Web-based  evalu¬ 
ation  of  device-selected  images:  (1)  Up  to  30  fields  of  view 
(FOVs)  are  selected  using  a  proprietary  algorithm  based  on  a 
probability  of  that  FOV  containing  cellular  abnormality.4  (2) 
FOVs  are  arranged  in  a  hierarchical  rank  order  based  on  this 
probability.  (3)  The  FOV  images  are  black  and  white  and  low 
resolution,  so  that  in  JPEG  compressed  format,  they  are  suit¬ 
able  for  rapid  digital  transmission  using  Internet  conduits  of 
modest  bandwidth.  This  image-capture  capability  had  initially 
been  engineered  into  the  FocalPoint  device  to  allow  for  the 
accurate  localization  of  FOVs  to  observers  during  subsequent 
manual  microscopic  review,  and,  therefore,  the  images  are 
linked  to  slide  location  (Image  II. 

In  our  previous  study,1  the  FOV  images  were  transmitted 
as  e-mail  attachments,  which  made  their  display  and  manipu¬ 
lation  cumbersome  and  inefficient.  The  image  attachments  for 
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each  case  had  to  be  bundled  and  sent  to  the  specific  electronic 
address  of  the  reviewer.  In  any  clinical  application,  the  result¬ 
ing  diagnoses  would  have  had  to  be  further  delivered  to  other 
specified  addresses  for  hierarchical  review  and/or  consulta¬ 
tion.  The  present  study  was  designed  to  ameliorate  those  limi¬ 
tations  by  using  a  secure  Web  site  and  software  that  permits  a 
variety  of  image  display  options. 

The  software  application  was  custom  designed  for  the 
efficient  display  and  manipulation  of  FOV  images,  the  retriev¬ 
al  of  demographic  and  device-generated  information,  report¬ 
ing  capabilities,  and  data  storage.  Demographic  data  and 
device-captured  images  from  individual  cases  were  uploaded 
to  a  remote  server,  queued,  and  made  available  to  assigned 
persons  for  review.  A  secure  Web  site  displays  the  interpre¬ 
tations  and  images  to  additional  individuals  with  access  for 
further  processing  or  reporting.  Such  a  system  allows  for 
the  following:  (1)  distribution  of  case  images  to  people  with 
diagnostic  expertise,  who  may  be  in  remote  locations;  (2)  less 
transportation  of  perishable  materials;  (3)  timely,  encrypted 
transmittal  of  device-generated  images  and  interpretations; 
(4)  communication  of  results  to  patients,  caregivers,  or  both, 
who  may  be  in  a  remote  location  or  in  transit;  and  (5)  efficient, 
economical  triage  of  patients  to  additional  clinical  evaluation, 
testing,  treatment,  or  any  combination  thereof. 

The  present  study  augments  the  scope  of  our  prior  study 
with  a  much  larger  population  of  cases,  use  of  SurePath  (Becton 
Dickinson/TriPath)  and  ThinPrep  (Cytyc,  Marlborough,  MA) 


liquid-based  samples,  more  diagnosticians,  and  more  reading 
locations.  Our  goal  was  to  test  the  reliability  of  Web-based 
image  review  for  clinically  meaningful  interpretative  groups. 
As  in  the  pilot  study,  the  goal  was  to  distinguish,  by  means 
of  primary  triage,  cases  that  require  prompt  intervention  or 
further  clinical  evaluation  from  those  that  do  not. 


Materials  and  Methods 

The  study  protocol  was  reviewed  and  approved  by  the 
institutional  review  boards  at  all  investigational  sites.  The 
study  population  consisted  of  a  set  of  liquid-based  SurePath 
and  ThinPrep  cervical  cytology  slides  from  reference  cases 
with  known  diagnoses  from  the  fdes  of  2  participating  institu¬ 
tions.  Interpretations  were  made  using  the  criteria  of  the  2001 
Bethesda  System.5 

An  interface  was  established  between  a  FocalPoint  GS 
gynecologic  screening  device  (Becton  Dickinson/TriPath) 
located  in  Redmond,  WA,  and  a  computer  server  located  in 
Cambridge,  MA,  using  custom-designed,  Web-based  image 
and  data  acquisition  and  review  software  (Consult,  Wellogic, 
Cambridge,  MA)  that  automatically  queries  the  FocalPoint 
database  at  regular  intervals  during  the  screening  process.  The 
software  was  developed  in  consultation  with  the  investiga¬ 
tors  to  address  the  specific  needs  of  the  project.  The  software 
application  is  based  on  an  existing  one  that  has  been  certified 


■Image  II  FocalPoint  gynecologic  screening  (GS)  reviewing  station  showing  a  black  and  white  image  of  the  field  of  view  (FOV) 
captured  in  the  upper  left  part  of  the  instrument  (box)  and  corresponding  microscopic  image  in  the  lower  right.  Note  the  FOV 
locations  illustrated  in  the  slide  schematic  at  the  bottom  of  the  Image  with  the  current  FOV  highlighted.  Also  note  the  square 
image  captured  by  the  device.  The  circular  FOV  forms  the  basis  for  the  device's  final  analysis  and  is  the  area  reviewed  in  the 
microscope  in  the  commercial  version  of  the  FocalPoint  GS  system. 
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as  compliant  with  the  Health  Insurance  Portability  and 
Accountability  Act  of  1996  for  use  in  patient  care  settings. 

Web  site-based  image  review  and  data  management 
software  were  developed  for  the  present  study  to  overcome 
the  limitations  inherent  in  the  use  of  e-mail  transmission  and 
case  review.  The  custom-designed  software  used  in  this  study 
incorporated  the  following  capabilities:  (1)  entry  of  clinical 
and  demographic  data  of  individual  cases  and  linkage  to  cap¬ 
tured  images;  (2)  upload  of  specimen-derived  images  that  are 


device  selected  from  an  automated  instrument  at  a  remote  site; 
(3)  encryption  of  transmitted  data  and  images;  (4)  secure  stor¬ 
age  of  case-specific  data  and  images;  (5)  orderly  presentation 
of  clinical  data  and  images  on  a  computer  monitor;  (6)  access 
at  the  Web  site  to  individual  cases  or  to  lists  of  cases,  by 
registered  cytotechnologists,  cytopathologists,  clinicians,  and 
patients;  (7)  presentation  of  thumbnail  FOV  images  in  rank 
order  (Image  21  and  (Image  3(;  (8)  full-screen  presentation  of 
any  1  image,  panels  of  4  or  9  enlarged  sequential  images  (with 
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(Image  2(  Cases  were  accessed  on  a  Web  site  with  retrieval  of  images  that  had  been  uploaded  to  a  remote  server  from  the 
automated  screening  device;  30  JPEG  images  with  the  highest  probability  of  abnormality  are  displayed  in  rank  order  (left),  with 
an  adjacent  scroll  tab.  A  panel  of  4  selected  highlighted  images  (fields  of  view  9-12)  is  enlarged  (right).  LSIL.  ThinPrep  slide. 


(Image  3(  Shown  are  30  JPEG  images  in  rank  order  according  to  probability  of  abnormality  (left),  enlarged  view  of  JPEG  image 
15  (right),  and  drop-down  view  of  specimen  details  (top),  including  distribution  quintile  and  adequacy.  High-grade  squamous 
intraepithelial  lesion  involving  glands.  ThinPrep  slide. 
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selection  of  the  “starting”  image)  (Image  2),  or  of  4  enlarged 
nonsequential  images  (with  point-and-click  selection  from 
thumbnail  images);  (9)  presentation  of  clinical  (age,  reproduc¬ 
tive,  and  menstrual  status)  and  device-generated  (distribution 
quintile,  computer-assessment  of  adequacy)  data  from  a  drop¬ 
down  folder  tab  on  the  monitor  (Image  3);  (10)  recording 
of  each  interpretation  on  every  case  by  a  cytotechnologist, 
cytopathologist,  and  (if  needed)  consultant;  (11)  electronic 
communication  between  any  or  all  people  responsible  for  any 
individual  case;  and  (12)  presentation  of  the  finalized  evalua¬ 
tions  of  selected  cases  to  clinicians,  patients,  or  both. 

Demographic  information  for  each  slide  was  entered 
manually  into  the  slide  review  database.  The  slides  were 
tagged  with  bar  codes,  stripped  of  other  identifiers,  and 
scanned  at  the  remote  site  on  the  FocalPoint  GS  system.  A  set 
of  up  to  30  of  the  highest  scoring  FOV  images  were  captured 
in  a  JPEG  compressed  format  (image  size,  12-16  KB  each). 
The  FocalPoint  GS  system  captures  only  black-and-white 
images,  each  of  which  corresponds  closely  to  an  approximate¬ 
ly  x200  microscopic  magnification  when  viewed  on  a  moni¬ 
tor.  Of  note,  the  image  captured  by  the  device  represents  only 
the  central  area  of  a  x200  magnification  FOV  that,  in  the  com¬ 
mercial  application  of  the  FocalPoint  GS  device,  is  intended 
to  be  viewed  as  a  full  circular  microscopic  FOV.  Hence,  each 
image  used  in  the  study  does  not  contain  the  entire  FOV 
intended  to  be  viewed  in  the  normal  operation  of  the  device. 
Sets  of  JPEG  images  for  each  case  were  transmitted  over  the 
Internet  and  uploaded  to  the  remote  server.  For  each  case,  30 
JPEG  images  were  device  generated  and  transmitted  unless 
fewer  FOVs  were  selected  by  the  scanner,  as  occurred  in 
22  cases  of  low  cellularity  (for  which  5-29  FOVs  [mean,  22 
FOVs]  were  selected).  To  capture  maximal  information  from 
the  abbreviated  FOVs,  30  images  were  selected  for  review;  in 
the  intended  use  of  the  device,  only  the  top  10  scoring  FOVs 
are  used.  The  maximum  file  size  per  case  was  less  than  500 
KB.  Known  demographic  information  (eg,  age  and  menstrual 
status)  and  standard  FocalPoint  device-generated  data  (dis¬ 
tribution  quintile,  computer  assessment  of  adequacy)  were 
entered  into  the  database  for  each  case. 

The  review  stations  were  the  standard  personal  comput¬ 
ers  (Microsoft  Windows  2000  Professional  operating  system, 
Microsoft  Explorer  browser,  Microsoft,  Redmond,  WA)  used 
by  the  reviewers  in  their  secure  institutional  networks,  aug¬ 
mented  by  high-resolution  19-inch  LCD  monitors  to  enhance 
the  clarity  of  images.  The  monitor  settings  and  image  display 
options  were  standardized  across  institutions.  The  study  did 
not  seem  to  be  affected  by  network  delays  at  either  institution. 
The  software  application  displayed  up  to  30  thumbnail  images 
for  each  case  in  a  panel  on  the  left  side  of  the  monitor  (Images 
2  and  3).  Each  individual  image  could  be  displayed  as  a  single 
large  image  or  as  a  panel  of  4  or  9  consecutive  images  at  less 
than  maximal  enlargement,  depending  on  the  preference  of 
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the  observer.  In  addition,  for  image  comparison  purposes,  any 
4  selected  nonconsecutive  FOVs  could  be  displayed  together 
as  a  panel  of  enlarged  images.  Demographic  and  FocalPoint 
data  for  each  case  were  accessed  by  using  a  drop-down  tab. 
Interpretations  for  each  case  could  be  entered  in  a  results  field 
after  image  review  and  forwarded  to  another  individual  for 
final  review  (“escalate”  tab),  or  submitted  as  a  final  diagnosis 
(“complete”  tab). 

Glass  slides  were  not  manually  reviewed  by  any  partici¬ 
pant  during  the  study,  and  the  reviewers  received  no  training 
in  the  evaluation  of  the  black-and-white  FOV  images.  Each 
of  the  study  cases  was  interpreted  independently  by  3  ASCP- 
registered  cytotechnologists  (D.P.B.,  L.B.,  and  S.B.B.)  and  3 
board-certified  cytopathologists  (B.A.C.,  J.H.E.,  and  D.C.W.) 
who  were  masked  to  the  reference  diagnoses.  During  the 
study,  an  off-line  worksheet  was  used  that  allowed  the  record¬ 
ing  of  participant  observations  in  a  format  intended  as  an 
aid  to  future  data  analysis  and  further  technique  refinement. 
Reviewers  annotated  the  reasons  that  a  given  interpretation 
was  favored  and  features  of  each  case  that  made  evaluation 
difficult.  For  every  case,  they  also  recorded  the  first  FOV  for 
which  an  abnormality  was  suspected,  the  first  FOV  at  which 
the  final  interpretation  became  fixed,  and  the  specific  FOVs 
that  contained  abnormalities  or  in  which  other  observations 
were  made.  Assessments  regarding  adequacy  and  any  other 
comments  about  each  case  were  recorded. 

Cases  were  interpreted  as  specifically  as  possible  by  the 
reviewers  and  then  assigned  to  1  of  2  categories:  (1)  nega¬ 
tive  for  intraepithelial  lesion  or  malignancy  (NILM;  normal 
findings,  reactive  changes,  organisms)  or  unsatisfactory  for 
evaluation;  or  (2)  any  cellular  abnormality  diagnostic  of  an 
intraepithelial  lesion  (or  worse)  or  worrisome  for  its  presence 
(high-grade  squamous  intraepithelial  lesion  [HSIL];  low-grade 
SIL  [LSIL];  atypical  squamous  cells,  cannot  exclude  high- 
grade  lesion  [ASC-H];  atypical  squamous  cells  of  uncertain 
significance  [ASC-US];  or  atypical  glandular  cells  [AGC]).  It 
was  intended  that  this  binary  categorization  would  determine 
if  the  combination  of  automated  screening  and  Web-based 
transmittal  and  presentation  of  images  could  be  used  to  triage 
patients  appropriately,  with  the  cutoff  point  representing  clini¬ 
cally  significant  lesions  requiring  further  evaluation. 

After  reviewers  had  recorded  their  initial  binary  assess¬ 
ments  for  each  case,  their  “trial”  interpretations  were  compared 
with  the  reference  diagnoses.  All  6  reviewers  met  at  1  location 
to  review  the  images  of  selected  cases  for  which  there  were  1 
or  more  discrepant  interpretations,  with  simultaneous  access 
to  the  images  of  these  cases  (from  the  Web  site)  and  written 
observations  that  they  had  made  at  the  time  of  masked  review 
recorded  in  a  computerized  database.  The  potential  sources  of 
error  were  discussed  at  this  time.  The  slides  of  cases  for  which 
all,  or  nearly  all,  reviewers  incorrectly  categorized  the  binary 
triage  compared  with  the  reference  diagnosis  were  screened 
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manually  by  the  investigators  to  confirm  or  change  a  reference 
diagnosis  for  data  analysis  purposes. 

A  comparison  was  done  of  reviewer  categorization 
(binary  triage)  vs  the  reference  diagnosis  for  each  case  and 
tabulated  based  on  how  many  reviewers  achieved  the  cor¬ 
rect  result.  The  numbers  of  reviewers  correctly  triaging  the 
cases  into  these  2  categories  were  also  tabulated  for  each 
Bethesda  diagnostic  group  for  the  reference  cases.  In  addi¬ 
tion,  rates  of  sensitivity  [True  Positive/(True  Positive  +  False 
Negative)]  and  specificity  [True  Negative/(True  Negative  + 
False  Positive)]  were  calculated  at  the  level  of  agreement  by  3 
or  more  reviewers  because  it  was  postulated  that  this  degree  of 
concordance  indicated  that  a  correct  interpretation  could  have 
been  made  in  each  case. 


Results 

The  study  set  included  101  SurePath  and  90  ThinPrep 
slides.  We  excluded  11  other  ThinPrep  slides  from  the 
study  because  they  could  not  be  adequately  scanned  by  the 
FocalPoint  device.  In  these  cases,  it  seemed  that  scanning  was 
impeded  by  technical  problems  such  as  coverslip  placement, 
artifacts  of  preparation,  and  staining  irregularities.  Reference 
diagnoses  included  the  following:  HSIL,  18;  LSIL,  63; 
ASC-H,  3;  ASC-US,  6;  AGC,  5;  NILM,  93;  and  unsatisfac¬ 
tory  for  evaluation,  3  ITable  II. 

Images  2  and  3  and  (Image  41  include  representative 
device-captured  FOV  images  from  ThinPrep  slides,  and  Hm- 


ITable  II 

Reference  Diagnoses  and  Preparatory  Method  for  191  Study 
Cases 


Bethesda  Category5 

SurePath  (n  =  101)* 

ThinPrep  (n  =  90)’ 

HSIL 

6 

12 

LSIL 

45 

18 

ASC-H 

0 

3 

ASC-US 

2 

4 

AGC 

0 

5f 

NILM 

48 

45 

Unsatisfactory  for 
evaluation 

0 

3 

AGC,  atypical  glandular  cells;  ASC-H,  atypical  squamous  cells,  cannot  exclude 
high-grade  lesion;  ASC-US,  atypical  squamous  cells  of  undetermined  significance; 
HSIL,  high-grade  intraepithelial  lesion;  LSIL,  low-grade  squamous  intraepithelial 
lesion;  NILM,  negative  for  intraepithelial  lesion  or  malignancy. 

*  One  slide  per  case. 

+  The  slides  from  these  cases  were  rescreened  after  the  trial;  3  of  5  were  diagnosed 
as  NILM  by  majorities  of  reviewers,  and  these  patients  had  subsequent  surgical 
procedures  that  were  negative. 


age  51,  llmage  61,  and  llmage  71  show  images  from  SurePath 
slides.  The  images  derived  from  ThinPrep  slides  tended  to 
have  fewer  cells  for  evaluation  in  each  FOV  than  those  pro¬ 
duced  from  SurePath  slides,  and  the  cells  in  the  images  from 
ThinPrep  slides  were  less  evenly  spread  across  the  FOV. 
These  findings  directly  relate  to  the  lower  cell  concentra¬ 
tion  and  increased  “open”  areas  routinely  present  on  each 
ThinPrep  slide  compared  with  the  higher  and  more  evenly 
distributed  cellularity  of  a  routine  SurePath  slide.  SurePath 
slides,  on  the  other  hand,  showed  fewer  well-focused  cells  and 
poorer  overall  image  resolution,  especially  in  hypercellular 


llmage  41  Glandular  cell  atypia  was  difficult  to  assess  on 
low-resolution  JPEG  images.  Adenocarcinoma  in  situ. 
ThinPrep  slide. 


■Image  51  Case  judged  negative  for  intraepithelial  lesion  or 
malignancy  with  abundant  normal  endocervical  cells.  The 
3  reviewers  not  accustomed  to  screening  SurePath  slides 
called  this  case  atypical. 


690  Am  J  Clin  Pathol  2008;  1 29:686-696 

DOI:  1 0.1 309/G  RAVI  6QP8JR5XTPF 


©  American  Society  for  Clinical  Pathology 


Informatics  /  Original  Article 


■Image  61  High-grade  squamous  intraepithelial  lesion  cell 
(center  right)  and  benign  cells  for  comparison.  Hypercellular 
fields  of  view  sometimes  had  poorer  resolution  of  the  nuclei 
of  atypical  cells.  SurePath  slide. 


samples.  This  finding  could  be  related  to  the  characteristic 
3-dimensional  depth  of  routine  SurePath  slides,  which  might 
have  led  to  more  variability  in  the  device-determined  plane 
of  focus. 

The  reviewers’  ability  to  correctly  assign  the  cases  to 
either  of  the  2  categories,  NILM/unsatisfactory  or  abnormal, 
is  shown  in  (Table  21.  Three  or  more  investigators  agreed  on 
the  correct  general  categorization  of  NILM/unsatisfactory  in 
89%  of  cases  (85/96)  and  abnormal  in  83%  of  cases  (79/95). 
This  corresponds  to  a  sensitivity  of  88%  and  a  specificity  of 
93%  for  binary  categorization  at  the  level  of  agreement  by 
at  least  3  reviewers.  The  ability  of  the  reviewers  to  triage 
individual  cases  to  the  abnormal  group  and  target  a  specific 
Bethesda  category  of  reference  diagnosis  is  shown  in  (Table 
31.  For  specific  Bethesda  interpretations,  the  frequencies  at 
which  3  or  more  reviewers  triaged  the  cases  to  abnormal  were 
as  follows:  ASC-US,  83%  (5/6);  ASC-H,  100%  (3/3);  LSIL, 


■Image  7(  Abnormal  cells  and  cell  clusters  are  not  always 
positioned  in  the  center  of  the  selected  field  of  view  (FOV) 
owing  to  the  full-field  interpretation  algorithmic  method  used 
in  the  FocalPoint  system.  Abnormal  cells  can  be  localized 
to  any  portion  of  the  selected  FOV.  Low-grade  squamous 
intraepithelial  lesion.  SurePath  slide. 


83%  (52/63);  HSIL,  94%  (17/18);  and  AGC,  40%  (2/5).  That 
is,  the  reviewers’  ability  to  triage  to  the  abnormal  group  the 
cases  with  possible  high-grade  squamous  lesions  (HSIL  or 
ASC-H)  was  better  than  their  ability  to  assign  low-grade 
squamous  lesions  (LSIL)  to  that  category.  A  majority  of  the 
reviewers  incorrectly  categorized  7  (16%)  of  45  SurePath 
LSILs  and  4  (22%)  of  18  ThinPrep  LSILs,  whereas  a  majority 
incorrectly  assigned  only  1  (6%)  of  18  HSILs.  These  obser¬ 
vations  are  consistent  with  the  known  sensitivity  data  of  the 
FocalPoint  screening  device.4-6 

The  data  in  Table  3  suggest  that  the  system  did  not  per¬ 
form  as  well  for  glandular  as  for  squamous  cell  abnormali¬ 
ties,  but  this  apparent  disparity  was  all  but  eliminated  when 
the  individual  cases  were  examined  more  closely.  When  the 
slides  (all  ThinPrep)  of  the  5  cases  of  AGC  were  reexamined 
by  the  reviewers  as  a  group,  3  cases  were  judged  to  have  an 
incorrect  reference  diagnosis  by  majorities  of  the  participants 


■Table  21 

Comparison  of  Test  and  Reference  Triage  Groups* 


No.  of  Reviewers  With  Correct  General  Categorization 

General  Category  of  Reference  Cases 

6 

5 

4 

3 

2 

1 

0 

Total 

NILM/unsatisfactory  for  evaluation 

35 

29 

9 

12 

6 

4 

1 

96 

Abnormal 

32 

30 

9 

8 

4 

8 

4 

95 

NILM,  negative  for  intraepithelial  lesion  or  malignancy. 
*  Data  are  given  as  number  of  cases. 
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■Table  31 

Reviewer  Triage  to  Abnormal  vs  Bethesda  Category  (All  Cases)* 


Bethesda  Category  of  Reference  Cases 

No.  of  Reviewers  Triaging  Cases  to 

“Abnormal”  Group 

6 

5 

4 

3 

2 

1 

0 

Total 

ASC-US 

0 

1 

2 

2 

1 

0 

0 

6 

ASC-H 

0 

0 

2 

1 

0 

0 

0 

3 

LSIL 

27 

18 

3 

4 

3 

5 

3f 

63 

HSIL 

1 

9 

6 

1 

0 

0 

1  + 

18 

AGC 

0 

1 

0 

1 

1* 

2* 

0 

5 

AGC,  atypical  glandular  cells;  ASC-H,  atypical  squamous  cells,  cannot  exclude  high-grade  lesion;  ASC-US,  atypical  squamous  cells  of  undetermined  significance;  FOV,  field  of 
view;  HSIL,  high-grade  intraepithelial  lesion;  LSIL,  low-grade  squamous  intraepithelial  lesion. 

*  Data  are  given  as  number  of  cases. 

t  Reexamination  of  FOVs  from  these  cases  revealed  no  convincing  abnormality,  indicating  device  failure  of  abnormality  localization  to  selected  FOVs. 

*  The  slides  from  these  3  cases  were  rescreened  after  the  trial  and  were  interpreted  as  negative  for  intraepithelial  lesion  or  malignancy  by  majorities  of  the  participants;  subsequent 
surgery  failed  to  detect  a  glandular  abnormality. 


and  to  have  shown  no  more  than  reactive  glandular  changes. 
In  each  of  these  3  cases,  no  glandular  lesion  was  identified 
after  subsequent  surgery  (cone  biopsy  and  endocervical  and 
endometrial  curettage,  2  cases;  hysterectomy,  1  case).  On  the 
initial  FOV  review,  2  of  these  cases  had  been  assigned  to  the 
NILM/unsatisfactory  group  by  5  reviewers  and  the  other  case 
by  3  reviewers. 

Of  the  remaining  2  AGC  cases,  1  patient  had  endometrial 
adenocarcinoma  found  by  subsequent  hysterectomy.  When 
the  device-selected  FOVs  of  this  case  were  reexamined,  how¬ 
ever,  no  convincing  atypical  glandular  cells  were  found.  This 
case  had  been  called  NILM  by  4  participants  and  ASC-US  by 
2.  Finally,  1  patient  with  a  reference  diagnosis  of  AGC  had 
endocervical  adenocarcinoma  in  situ  in  a  subsequent  hysterec¬ 
tomy  specimen.  This  case  (Image  4)  had  been  called  abnormal 
by  5  of  6  participants  after  FOV  review,  but  there  was  little 
agreement  on  specific  categorization:  2  called  it  cancer  and  1 
each  called  it  LSIL,  ASC-US,  or  AGC.  Thus,  of  only  2  true 
AGC  cases  in  the  study,  one  lacked  abnormal  cells  in  device- 
selected  FOV  images  and  was  missed  on  this  basis,  and  the 
other  had  abnormal  cells  in  the  FOVs  and  correct  binary  triage 
by  5  of  6  reviewers.  Differences  in  the  specific  categories  to 
which  this  latter  case  was  assigned  may  be  related  to  difficul¬ 
ties  in  resolving  fine  details  in  small  glandular  cells. 

In  4  cases  with  a  squamous  abnormality  (SurePath 
LSIL,  2;  ThinPrep  HSIL,  1;  ThinPrep  LSIL,  1),  none  of  the  6 
reviewers  gave  an  interpretation  of  abnormal.  Reexamination 
of  the  30  device-selected  FOV  images  of  each  of  the  4 
cases  revealed  no  convincing  squamous  abnormalities.  The 
cases  screened  from  SurePath  slides  (both  containing  LSIL) 
showed  rare  (1  or  2)  dense  blotches  on  the  FOVs;  these  may 
be  koilocytotic  nuclei,  but  there  was  insufficient  detail  to  be 
sure.  In  the  cases  from  ThinPrep  slides,  subsequent  biopsies 
confirmed  the  presence  of  squamous  dysplasia  that  was  severe 
in  one  case  and  mild  in  the  other.  The  ThinPrep  HSIL  case 
appeared  to  contain  many  small  cells  of  parabasal  or  metaplastic 
type,  and  it  is  possible  that  similarly  sized  dysplastic  cells 
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may  have  been  missed  at  the  level  of  resolution  of  the  FOV 
images.  These  4  cases  indicate  that  the  abnormal  cells  in 
squamous  lesions  (particularly  LSIL)  may  not  be  captured  in 
the  rectangular  images  captured  by  the  automated  device  from 
the  selected  FOV. 

The  success  of  binary  triage  by  the  reviewers  is  sepa¬ 
rately  shown  by  preparatory  technique  in  ITable  41  and 
■Table  51  for  the  abnormal  cases.  There  were  differences  in 
the  composition  of  abnormal  cases  in  these  2  study  popula¬ 
tions,  how  and  where  the  reference  sets  were  compiled,  and 
the  availability  of  follow-up  data  from  subsequent  procedures. 
Of  53  abnormal  SurePath  slides,  45  (85%)  showed  LSIL  and 
none  showed  AGC;  follow-up  data  were  not  readily  avail¬ 
able  for  some  of  these  cases.  There  were  fewer  cases  with 
ThinPrep  slides  (11  cases  were  initially  eliminated  from  the 
study  because  their  slides  could  not  be  adequately  scanned  by 
the  FocalPoint  device),  but  among  the  42  abnormal  cases  in 
this  set  of  slides,  there  was  a  broader  distribution  of  Bethesda 
groups,  including  18  LSIL  (43%),  12  HSIL  (29%),  5  AGC 
(12%),  4  ASC-US  (10%),  and  3  ASC-H  (7%)  cases.  In  this 
population,  follow-up  data  from  subsequent  surgical  proce¬ 
dures  were  available  for  all  cases  with  abnormal  cytology. 
Moreover,  the  reviewers  differed  in  their  experience  with  each 
of  the  2  liquid-based  methods:  reviewers  at  one  institution  had 
experience  with  both  types  of  slides  but  had  been  routinely 
using  SurePath  slides  in  clinical  practice,  whereas  reviewers 
at  the  other  institution  had  little  or  no  recent  experience  with 
SurePath  slides,  having  used  the  ThinPrep  method  exclusively 
in  their  practice. 

As  in  our  previous  study,1  FOV  images  displayed  on  a 
computer  monitor  showed  poorer  resolution  for  a  given  degree 
of  magnification  than  would  have  been  achieved  by  standard 
light  microscopy.  Their  plane  of  focus  is  set  by  the  automated 
instrument  and  cannot  be  manipulated.  These  features  and  the 
lack  of  a  full-color  spectrum  made  the  evaluation  of  fine  or 
subtle  nuclear  and  cytoplasmic  detail  more  challenging  than 
in  light  microscopy,  particularly  for  the  smaller  cells  within  a 
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■Table  41 

Reviewer  Triage  to  Abnormal  vs  Bethesda  Category  (SurePath  Cases)* 


Bethesda  Category  of  Reference  Cases 

No.  of  Reviewers  Triaging  Cases  to  ' 

‘Abnormal”  Group 

6 

5 

4 

3 

2 

1 

0 

Total 

ASC-US 

0 

1 

1 

0 

0 

0 

0 

2 

ASC-H 

0 

0 

0 

0 

0 

0 

0 

0 

LSIL 

21 

12 

2 

3 

3 

2 

2f 

45 

HSIL 

0 

3 

3 

0 

0 

0 

0 

6 

AGC 

0 

0 

0 

0 

0 

0 

0 

0 

AGC,  atypical  glandular  cells;  ASC-H,  atypical  squamous  cells,  cannot  exclude  high-grade  lesion;  ASC-US,  atypical  squamous  cells  of  undetermined  significance;  FOV,  field  of 
view;  HSIL,  high-grade  intraepithelial  lesion;  LSIL,  low-grade  squamous  intraepithelial  lesion. 

*  Data  are  given  as  number  of  cases. 

+  Reexamination  of  FOVs  from  these  cases  revealed  no  convincing  abnormality,  indicating  device  failure  of  abnormality  localization  to  selected  FOVs. 

■Table  51 

Reviewer  Triage  to  Abnormal  vs  Bethesda  Category  (ThinPrep  Cases)* 


No.  of  Reviewers  Triaging  Cases  to  “Abnormal”  Group 


Bethesda  Category  of  Reference  Cases 

6 

5 

4 

3 

2 

1 

0 

Total 

ASC-US 

0 

0 

1 

2 

1 

0 

0 

4 

ASC-H 

0 

0 

2 

1 

0 

0 

0 

3 

LSIL 

6 

6 

1 

1 

0 

3 

1f 

18 

HSIL 

1 

6 

3 

1 

0 

0 

1f 

12 

AGC 

0 

1 

0 

1 

1* 

2* 

0 

5 

AGC,  atypical  glandular  cells;  ASC-H,  atypical  squamous  cells,  cannot  exclude  high-grade  lesion;  ASC-US,  atypical  squamous  cells  of  undetermined  significance;  FOV,  field  of 
view;  HSIL,  high-grade  intraepithelial  lesion;  LSIL,  low-grade  squamous  intraepithelial  lesion. 

*  Data  are  given  as  number  of  cases. 

t  Reexamination  of  FOVs  from  these  cases  revealed  no  convincing  abnormality,  indicating  device  failure  of  abnormality  localization  to  selected  FOVs. 

*  The  slides  from  these  3  cases  were  rescreened  after  the  trial  and  were  interpreted  as  negative  for  intraepithelial  lesion  or  malignancy  by  majorities  of  the  participants;  subsequent 
surgery  failed  to  detect  a  glandular  abnormality. 


given  image.  Each  cytologist  reported  spending  approximate¬ 
ly  1  to  2  minutes  per  case  on  average  to  review  the  images  and 
make  an  interpretation,  although  no  attempt  was  made  to  do  a 
formal  time  analysis  in  this  first-phase  study. 


Discussion 

In  the  United  States,  the  incidence  and  mortality  rates  of 
cervical  cancer  have  decreased  50%  and  70%,  respectively, 
in  the  50  years  since  widespread  screening  programs  were 
instituted.7,8  Other  countries  have  also  benefited  greatly 
from  population-based  screening  programs.9  Worldwide, 
however,  cervical  cancer  continues  to  be  a  leading  cause 
of  death  for  young  women  who  reside  in  countries  without 
systematized  screening  programs.10  These  regions  lack  an 
adequate  pool  of  trained  specialists  to  screen  and  interpret 
specimens.  They  typically  lack  funding  for  evolving  new 
technologies  such  as  liquid-based  preparations,  automated 
screening  devices,  and  human  papillomavirus  testing.  Even 
in  the  United  States,  laboratories  experience  fluctuating 
workloads  out  of  synchrony  with  their  available  workforce, 
resulting  in  specimen  backlogs. 

Remote  interpretation  of  automated  device-selected 
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digitized  images  transmitted  over  the  Internet  has  the 
potential  to  provide  effective  screening  and  clinical  triage 
to  individuals  in  these  populations.  “Proof  of  concept” 
was  provided  by  a  recent  feasibility  study  in  which  it  was 
shown  that  low-resolution  images  selected  by  an  automated 
screening  device  and  transmitted  over  the  Internet  as  e-mail 
attachments  could  be  adequately  interpreted  at  a  distant  site 
without  manual  microscopy,  using  primary  triage  methods 
for  clinically  meaningful  diagnostic  categories.1  An  algo¬ 
rithm  of  remote  review  of  a  laboratory’s  total  cervical  cytol¬ 
ogy  workload  with  triage  of  the  most  potentially  abnormal 
cases  to  on-site  review  by  the  originating  laboratory  has  the 
potential  to  select  the  “needles  out  of  the  haystack”  for  expe¬ 
dited  review.  In  addition,  laboratories  could  more  effectively 
share  personnel  resources  across  distances  without  resorting 
to  the  shipment  of  glass  slides,  thereby  easily  redistributing 
the  workload  among  available  resources. 

The  use  of  customized  Web  site-based  data  management 
software  facilitated  the  accommodation  of  more  reviewers 
and  reviewing  stations  in  the  present  study  than  there  had  been 
in  the  previous  one,  thereby  permitting  the  evaluation  of  a 
much  larger  population  of  cases.  Multiple  investigators  could 
evaluate  each  of  the  cases  at  their  own  particular  location,  in 
any  order,  at  their  own  pace,  and  with  as  many  “visits”  to  the 
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cases  as  needed. 

Although  the  FocalPoint  system  is  approved  by  the  US 
Food  and  Drug  Administration  (FDA)  for  automated  scanning 
of  SurePath  slides,  it  can  also  be  reconfigured  for  the  assess¬ 
ment  of  ThinPrep  slides,  for  which  it  has  shown  screening 
efficacy.11  It  is  FDA  approved  for  the  screening  of  conven¬ 
tional  smears  as  well,  but  the  present  study  was  designed 
to  evaluate  slides  made  from  liquid-based  preparations.  We 
included  both  liquid-based  slide  types  in  the  present  study,  not 
to  compare  the  desirability  of  one  method  over  the  other,  but 
to  show  that  both  can  be  used  in  such  a  system  and  to  judge 
the  special  features  of  each  type  of  preparation  when  evalu¬ 
ated  on  the  monitor-based  FOV  reviewing  platform.  This  was 
necessary  because  we  anticipated  that  one  or  the  other  type 
of  liquid-based  preparation  may  be  favored  or  in  use  in  the 
target  population  of  our  subsequent  studies  or  their  intended 
applications.  Differences  in  the  images  derived  from  the  2 
types  of  samples  were  seen.  ThinPrep  slide  images  tended  to 
have  fewer  cells  per  FOV  than  those  from  SurePath  slides, 
and  the  cells  were  distributed  less  uniformly  across  each 
FOV.  The  images  produced  from  SurePath  slides,  however, 
had  poorer  image  resolution,  especially  in  cellular  samples, 
generally  because  a  thicker  layering  of  cells  in  these  prepa¬ 
rations  produced  more  variability  in  the  device-determined 
plane  of  focus. 

Even  though  image  quality  and  size  were  augmented  by 
the  use  of  larger,  flat-screen  monitors  with  better  resolution 
than  the  monitors  used  in  our  prior  study,  the  resolution  of  the 
FOV  images  still  was  not  as  optimal  as  that  of  standard  light 
microscopy,  and  the  fixed  plane  did  not  allow  manipulation 
of  cells  that  were  not  optimally  focused  when  captured.  In  the 
present  study,  these  limitations  may  have  hindered  the  appre¬ 
ciation  of  fine  cellular  details  in  glandular  cells,  which  tended 
to  be  small  and  to  present  in  closely  packed  groups  (Image 
4),  which  led  to  difficulty  in  classifying  a  case  of  adenocarci¬ 
noma  in  situ  (although  it  was  recognized  as  abnormal  by  5  of 
6  reviewers).  Conversely,  3  investigators  working  at  hospitals 
that  screen  only  ThinPrep  slides  tended  to  overcall  glandular 
atypia  from  images  derived  from  SurePath  slides  that  had 
abundant  normal  endocervical  cells  (Image  5). 

Despite  a  poorer  focus  and  resolution  of  images  derived 
from  some  SurePath  slides  that  were  hypercellular  (because 
of  a  thicker  layer  of  cells  per  FOV),  HSIL  was  correctly 
recognized  despite  the  small  size  from  cellular  features  unaf¬ 
fected  by  these  limitations,  such  as  nuclear/cytoplasmic  ratio, 
nuclear  shape,  and  degree  of  hyperchromasia,  compared  with 
neighboring  cells  in  the  FOV  image  (Image  6).  The  reviewers 
noted  that  LSIL  cells  were  easier  to  recognize  than  HSIL  cells 
because  of  their  larger  nuclear  dimensions  and  the  occasional 
presence  of  human  papillomavirus-associated  cytopathic 
perinuclear  halos. 

The  most  abnormal  cells,  however,  were  not  always 
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positioned  near  the  center  of  the  FOV  (Image  7),  nor  were 
they  always  present  in  the  FOVs  that  had  been  assigned  the 
highest  probability  scores  for  having  cellular  abnormality. 
These  observations  relate  to  FocalPoint  system  program¬ 
ming  because  the  device  selects  FOVs  with  the  highest 
probabilities  of  cellular  abnormality  based  on  its  assessment 
of  the  field  “as  a  whole”  and  not  of  individual  cells  within 
the  field  or  their  position  within  it.  Moreover,  the  computer 
algorithm  used  to  assign  these  probabilities  has  an  intended 
bias,  based  on  training  and  prioritization,  toward  the  detec¬ 
tion  of  high-grade  squamous  lesions  over  the  identification 
of  other  types  of  abnormality. 

In  follow-up  group  discussions,  it  seemed  that  some 
reviewers  had  been  more  bothered  than  others  by  the  lack  of 
a  full-color  spectrum,  particularly  in  the  evaluation  of  subtle 
differences  in  cytoplasmic  characteristics  of  squamous 
metaplastic  and  high-grade  squamous  dysplastic  cells,  and 
that  some  investigators  were  more  adept  than  others  in 
making  cellular  and  nuclear  size  comparisons  from  FOV 
images.  Different  reviewers  developed  their  own  preferred 
strategies  of  approach  to  individual  cases  using  the  display 
software,  which  probably  evolved  between  review  of  early 
and  later  cases. 

The  reviewers’  success  in  binary  triage  of  the  cases  was 
analyzed  at  the  level  of  agreement  of  3  or  more  individuals 
(half  the  total  number  of  viewers).  This  level  of  agreement 
was  selected  for  this  first-phase  trial  because  most  of  the 
reviewers  had  no  experience  in  the  evaluation  of  black-and- 
white  FOV  images,  none  had  used  the  Web-based  image 
display  software,  no  prior  training  in  FOV  assessment  had 
been  given,  and  some  of  the  reviewers  were  familiar  with 
one  liquid-based  preparation  but  not  sufficiently  with  the 
other.  It  was  hypothesized  that  if  3  or  more  of  6  observ¬ 
ers  were  able  to  classify  a  case  correctly,  there  was  most 
likely  good  reason,  and,  hence,  training  would  improve 
performance  of  the  remaining  3  or  fewer  observers.  At  this 
level  of  agreement,  sensitivity  of  88%  and  specificity  of 
93%  were  obtained  for  the  categorization  of  the  cases  into 
1  of  the  2  groups.  When  various  threshold  levels  of  abnor¬ 
mality  were  separately  examined  for  cases  with  squamous 
abnormalities,  the  sensitivity  of  detection  of  HSIL  or  worse 
(94%)  was  better  than  the  sensitivities  at  lower  threshold 
levels  (85%  for  LSIL  or  worse  and  86%  for  ASC-US  or 
worse)  ITable  61.  These  figures  compare  favorably  with 
reports  of  the  sensitivity  of  manual  screening  reported  in 
clinical  trial  settings.  In  the  original  FDA  clinical  trials 
for  the  FocalPoint  screening  system,  overall  sensitivity  for 
binary  triage  of  all  abnormal  cases  (ASC-US  or  worse)  was 
78%,  for  LSIL  or  worse  was  83%,  and  for  HSIL  or  worse 
was  93%, 4  figures  that  are  very  close  to  the  results  obtained 
in  this  study. 

One  limitation  of  the  present  study  is  the  fact  that 
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■Table  61 

Sensitivity  of  Detection  of  Various  Squamous  Abnormality 
Thresholds  by  at  Least  Three  of  Six  Reviewers 


Abnormality  Threshold 

Sensitivity  (%) 

ASC-US  or  worse 

86 

LSIL  or  worse 

85 

HSIL  or  worse 

94 

ASC-US,  atypical  squamous  cells  of  undetermined  significance;  HSIL,  high-grade 
intraepithelial  lesion;  LSIL,  low-grade  squamous  intraepithelial  lesion. 


abnormal  cases  were  overrepresented  in  the  current  study 
population  compared  with  the  aforementioned  prospective 
study.4  This  difference  could  enhance  the  apparent  sensitiv¬ 
ity  of  detection  owing  to  increased  disease  prevalence  and, 
hence,  heightened  vigilance  of  observers.  A  prospective 
study  to  more  thoroughly  test  these  operating  characteris¬ 
tics  is  underway  at  the  time  of  this  writing.  When  specific 
types  of  cellular  abnormality  were  examined,  it  seemed 
that  reviewers  were  better  at  recognizing  as  abnormal 
the  cases  that  contained  potential  high-grade  squamous 
lesions  than  cases  with  low-grade  squamous  abnormalities. 
Identification  of  glandular  lesions  remains  a  challenge  to 
the  system,  as  in  the  standard  manual  review  process;  how¬ 
ever,  these  lesions  generally  represent  a  small  minority  of 
cases  in  actual  practice. 

We  anticipate  that  the  interpretation  of  FOVs  can  be 
improved  with  training  and  the  development  of  strategies 
to  compensate  for  the  difficulties  of  FOV  assessment  in 
particular  types  of  cases.  The  identification  of  common 
and  recurring  pitfalls  in  evaluation  and  of  problematic 
types  of  cases  will  inform  us  in  the  assembling  of  study 
sets  for  the  purpose  of  training.  Can  the  quintile  data 
provided  by  the  instrument  be  used  as  an  ancillary  clue  in 
individual  cases?  In  separate  studies,  we  are  investigat¬ 
ing  the  optimal  number  of  FOVs  for  effective  specimen 
triage  and  whether  determinations  of  specimen  adequacy 
can  be  made  reliably.  It  also  needs  to  be  established  if, 
and  how  often,  diagnostic  cells  are  present  in  the  device- 
selected  FOV  (which  is  circular)  but  not  in  the  captured 
IPEG  image  (which  is  rectangular  and  a  centrally  located 
subset  of  the  larger  circular  field).  For  example,  in  the  4 
squamous  lesions  (3  LSIL  and  1  HSIL)  that  were  missed 
by  all  reviewers,  the  abnormal  cells  were  present  on  the 
slides  but  not  in  the  rectangular  JPEG  images.  Were  they 
present  at  the  periphery  of  the  device-selected  LOVs  but 
outside  the  boundaries  of  the  JPEG  image?  In  the  future, 
additional  avenues  of  investigation  might  include  the 
addition  of  color  image  capture,  wider  FOVs,  and  the 
making  of  virtual  slide  images  for  whole-slide  manipu¬ 
lation  and/or  focusing.  Possibly,  virtual  microscopy  or 
whole-slide  imaging  might  be  used  to  identify  solutions  for 


the  shortcomings  of  static  digital  image  transmission. 

We  are  encouraged  that  Web-based  interpretation  of 
automated  device-selected  images  transmitted  electroni¬ 
cally  can  permit  optimization  of  interpretative  expertise  and 
potentially  provide  cytology  screening  services  to  popula¬ 
tions  that  currently  have  none,  enhance  resource  sharing 
between  laboratories,  and  centralize  cytology  interpretation 
services.  Reviewers  will  be  further  trained  from  study  sets 
acquired  from  the  present  exercise  in  an  attempt  to  further 
enhance  the  capability  of  interpreting  FOV  images  in  this 
system.  In  addition,  a  larger  prospective  trial  with  speci¬ 
mens  from  current  patients  is  underway. 
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